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ABSTRACT 

In the Information Communication Technology era teachers will have to wisely use the 
online environment in order to realize a new pedagogy. The penetration of the internet 
and collaborative online instruments to teaching and learning affect the quality of 
teaching. We have developed a digital indicator evaluate the quality of the online tasks 
in terms of content knowledge, pedagogical knowledge, technological knowledge and 
technological pedagogical content knowledge (TPACK). The score for the task is 
obtained immediately, with emphasis of its weaknesses and strengths. The indicator 
was used to examine 53 online tasks that were created by teachers from 2001 to 2007. 
We found that teachers can use this indicator for evaluating the quality of the tasks 
that were developed as well as to test the improvement that took place in their tasks 
over time. 

Keywords: Digital indicator. Online tasks, TPACK, Evaluation, Technology knowledge, 

IT skills. Professional development 

INTRODUCTION 

Life in the computerized world, which is characterized by the penetration of 
information technologies and which changes rapidly, is affected in many fields. The 
penetration of the internet into different fields of teaching and learning may have a 
significant effect on the quality of teaching. Numerous researchers claim that time is 
needed in order to examine the real effects of technology on pedagogy (Gao et al. 
2006; Hui et al. 2005) and on the teachers in the field "to harness the horses of 
technology to the cart of pedagogy" (Noski et al. 2006). However, the new learning 
environment must be created on a rational pedagogical basis and not only on a 
technological basis (Salomon 2000).The theoretical part of this article describes online 
learning, the characteristics of knowledge required of teachers who integrate 
technology in teaching, the characteristics of an online task, and various indicators for 
evaluating an online task. Because we did not find a dynamic and digital indicator for 
evaluating the quality of online tasks which encompasses all of the aspects which in 
our opinion should be expressed in an indicator, such as PK, PCK, TK and TPACK, which 
is actually the space expected to be expressed in an online task, we developed a digital 
indicator that evaluates the quality of online tasks. 

The goal was to develop a unique indicator that would reflect the quality of online 
tasks with reference to technological pedagogical content knowledge (TPACK). 
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The indicator is described in detail together with the considerations that were taken 
into account during its construction. The processes by which its validity and reliability 
were tested are also described. 

THEORETICAL FRAMEWORK 

Mishra and Koehler (2006, 2009) talk about teachers in the online environment, i.e. 
teachers who develop online learning environments must have technological 
knowledge as well as technological pedagogical knowledge. The online environment 
has the additional advantage of a high potential for learning and enrichment of the 
user also outside the school (Parsad et al. 2005). Online teaching has the potential for 
supporting meaningful learning, in which the student is active and acquires high-order 
cognitive skills such as carrying out generalizations, asking questions, expressing a 
well-argued opinion, making comparisons or solving problems (Capper, 2003; 
Herrington et al. 2005; Linn et al. 2004). Salomon (2000) presents a vision in which use 
of technological tools will comprise a lever for the implementation of meaningful 
learning, where technology will be at the disposal of pedagogy and will help in its 
realization. The integration between a pedagogical rationale and technological tools 
enables the exposure of the student to rich and diverse information, enables dealing in 
complex contents that are relevant to the student and enables experience in 
constructing rich and original products based on the newly acquired knowledge. This is 
learning that encourages cooperation through a meaningful educational dialogue with 
the teacher and with peers, via the technological tools (Roschelle 2000; Lehtinen et al. 
1998). Technology also enables support of the learning process by means of 
intertwined evaluation that improves the process and directs the student's activity 
(Dori 2003; Shepard 2000). 

KNOWLEDGE ATTRIBUTES OF TEACHERS 
WHO INTEGRATE TECHNOLOGY IN TEACHING 

The concept technological pedagogical content knowledge (TPACK) expresses the 
space created between the technological knowledge, the pedagogical knowledge and 
the content knowledge. 
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Figure: 1 

TPACK - the space created between the technological knowledge, the pedagogicalknowledge 
and the content knowledge (Mishra and Koehler 2006, 2009) 
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This concept is based on Shulman's concept of PCK - pedagogical content knowledge. 
Shulman talks about the attributes of knowledge required of teachers who integrate 
technology in teaching, while referring to the complexity and the knowledge of the 
teachers in their field. 

Figure.l describes the complexity and the relations between content knowledge (CK), 
pedagogical knowledge (PK) and technological knowledge (TK). The correct 
combination of technology in pedagogy in a particular subject must take into account 
the dynamic combination between the components and the intersections between 
them. 

A teacher who can navigate between these interrelations represents an expert who is 
different from an expert only in the disciplinary field of knowledge, only in the 
technology field of knowledge or only in the pedagogical field of knowledge (Mishra 
and Koehler 2006, 2009). 

CHARACTERISTICS OF AN ONLINE TASK 

One of the school's goals is to afford a pedagogical answer to the needs of the internet 
generation students. The learners' tasks should be relevant, innovative and attractive, 
thus increasing the relevance of the school for the learners and for society after a long 
time in which it decreased, since the early 1980s (Condition of Education, 2002). 

An online learning activity is a learning activity in which online tools found in an online 
environment are exploited and can deal in a content field from the school curriculum. 
The online task must combine the development of thinking and computerized 
information skills such that the learners will make effective, wise and valuable use of 
the internet for their learning (Rotem and Peled 2008). 

The online task must be dynamic, challenging to the student and enable understanding 
the contents by creating the proper connections (Selant 2007). The complexity of the 
online learning task (beginning with a simple worksheet on the internet) is derived 
from the task's pedagogical goals. The online activity can be carried out by personal or 
group learning, where the learner experiences different learning than in the traditional 
lesson. 

Online learning is not intended to replace traditional learning, but rather to 
demonstrate a new, interesting, challenging aspect that affords the learner a 
possibility for developing independent learning connected to the infinite information on 
the internet, while knowing what is important. The concept of an online learning task is 
a generalized name for the conduct of a lesson on the web as independent individual 
learning or as team learning (Rotem and Peled 2008). 

The online learning task is realized in a learning environment which has online tools via 
the internet and ensures: 

> Accessibility to information (human or technological); 

> Retrieval of information according to context and need; 

> Interpersonal communication for purposes of learning and personal 
empowerment, which is not limited to place and time; 

> Reception of information in a digital text in a diversity of possible 
modalities; 

> Creative information output which reflects the knowledge acquired and 
created in the learner, via a digital text. 
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Or Meir (2005) defined the characteristics of the online learning task that in her 
opinion should be in every online learning task: Congruence with the school 
curriculum; Adaptation of the task to the learners' attributes; The skills required of the 
student are clearly defined and are achievable; Time: the task can be completed during 
the school day or at home; Place: the task can be completed within the school walls or 
outside the school; Organization of learners: the task can be performed individually or 
by teamwork and referral to online sites and materials; 

In her opinion, the components of the online task are: Title and subject of the task; 
Clear and achievable goals; Item of information that contains basic knowledge on the 
task's subject; Short description of the task and what is required of the learner; 
Explanations for performing the task from the content aspect and from the 
technological aspect; Mode of communication with the teacher/peers/experts - email, 
forum or chat; Links to relevant sources of information on the internet; Clear definition 
of the product expected of the student and suggestions for different ways of evaluating 
the process and the product. 

INDICATORS FOR EVALUATING ONLINE TASKS 

An indicator is a systematic and focused tool that reflects, sets, details and organizes 
the measures and criteria for evaluating. Use of an indicator attempts to reduce the 
disadvantages of subjective evaluation instruments without losing their advantages 
(Smith 2004). The indicator indicates what is taken into account and what determines 
the grade (Goodrich 1997; Wilson 1994). Accordingly, the indicator teaches what is 
evaluated and enables learning the criteria for optimal evaluation. There exists 
difficulty in ensuring reliable, fair and valid judgment of complex, multi-dimensional 
tasks with several response possibilities. The indicator affords a solution for this 
difficulty: in order to obtain fruitful feedback on the learning and teaching process, and 
in order that the evaluation will be reliable, valid, constructive and fair to the student, 
it is important to rely on a multi-dimensional evaluation indicator which contains 
detailed criteria according to which the student will be evaluated regarding a particular 
task (Birenboim 1997). 

Online learning is essentially different from face-to-face learning and obligates 
different methodologies for evaluating learning processes. Within the various in- 
service training frameworks, we did not find indicators that encompass and exhaust 
the measures which in our opinion should be included in an online task. There exist 
various indicators that evaluate online tasks, but we did not find an indicator that was 
validated and examines the measures and criteria necessary for turning an online task 
into a task that meets all the requirements at their best, i.e. an indicator that evaluates 
the space created in a task between technological content knowledge, pedagogical 
knowledge and content knowledge. 

For example Or Meir (2005) presents a list of the characteristics of an online task. 
When evaluating a task according to Or Meir, one must indicate only whether or not a 
component exists in the task. The components she indicates are: "integrative task that 
requires performing connections between separate parts of the curriculum; task that 
enables different solutions and evokes pluralistic thinking; task in which the learners 
must identify and formulate the problem/dilemma which is presented; task that 
enables the learners to work together out of a need that stems from the scope of the 
task and from the opportunity to present several points of view; task that requires 
planning an expected course of performance; task that enables choice for the learners, 
making decisions and control of their work; task that obligates the learners to use 
online databases and different sites on the internet; task that develops know-how 
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skills in the learners; task that connects to the learners' real world, has meaning for 
them, where they required to solve problems that are real in their reality of life; task in 
which the learners are required to carry out integration between prior and new 
knowledge and task that affords an opportunity for self-evaluation and reflection." 

Another example of an indicator that evaluates an online task is an indicator which 
includes six measures (exploiting technology, the required level of thinking, the extent 
of collaboration, the closeness of the contents to the learners' world, direction towards 
rich products and intertwined evaluation). Such an indicator comprises a measure of 
good teaching, according to the social-constructivist worldview. Three levels of 
performance were defined for each measure: At the highest level of performance the 
teacher acts according to the principles of the social-constructivist approach and 
exploits the added value of technology for pedagogy. 

At the medium level of performance there exists an attempt to exploit the added value 
of technology for pedagogy, but the activities do not indicate a fundamental change in 
teaching. At the low performance level technology is used as an instrument that 
supports the traditional teaching and evaluation methods (Inbal-Shamir and Kelly 
2008). An example of a more detailed indicator that includes three levels of 
performance includes the following measures: point of origin for the task, use of the 
internet, adaptation of the activity to the properties of the tools, thinking levels, 
solution for heterogeneity, textual design, use of internet communications tools, 
contribution of the computer to the learning process, definition of the product, 
correlation between the product and the task, adaptation of the learning product to the 
computerized tools and mediation of the task. The three levels of performance for 
these measures are: usually not, usually and optimal. There is no quantification for 
each level of performance (Nashi and Doron 2008). Cohen et al. (2008) developed a 
collaborative digital "computerized performance meter" indicator. This indicator is 
unique in that several evaluators evaluate the same online task. 

In the collaborative performance meter each criterion must be ranked on a scale 
between 0 and 4. There is no detailing of the performance levels of each of the scales 
and there is no function that calculates the final score for the task. 

In our opinion, these indicators (Cohen et al. 2008; Nashi and Doron 2008; Or Meir 
2005) lack additional criteria from the field of content knowledge and the field of 
pedagogical content knowledge and/or do not have clear performance levels. Indicator 
for evaluating improvement of the online tasks We did not find a dynamic and digital 
indicator for evaluating the quality of online tasks which encompasses all of the 
aspects which in our opinion should be expressed in an indicator, such as pedagogical 
knowledge, pedagogical content knowledge, technological knowledge and pedagogical 
technological content knowledge. 

Therefore, we developed a digital indicator that evaluates the quality of online tasks. 
The indicator was constructed based on an instrument for characterization and 
evaluation of online tasks (Rotem 2006). 

DESCRIPTION OF THE DIGITAL INDICATOR 

The digital indicator evaluates and characterizes a task according to compulsory and 
optional measures and according to internal criteria. The measures are: components of 
the learning task, pedagogical knowledge, pedagogical content knowledge, 
technological knowledge and technological pedagogical content knowledge (Table: 1). 
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Table: 1 

Characteristics of the indicator for evaluating the quality of an online task 


Measures 

Compulsory 

Components of the learning task 

General didactics (pedagogical knowledge) 

Pedagogical content knowledge 

Implementation of teaching in an online environment 
(technological knowledge and technological pedagogical 
knowledge) 

Originality and representations 

Optional 

Collaborative learning 

High-order cognitive skills and affording a solution for 
heterogeneity 

Use of the online task 


The compulsory measures include the components of the learning task, general 
didactics (pedagogical knowledge), pedagogical content knowledge and 
implementation of teaching in an online environment (technological knowledge and 
technological pedagogical content knowledge). 

The components of the learning task deal in the pedagogical field: the framework of 
the task, details of the prior knowledge required of the student, the learning goals in 
the content field and the goals of skills of using an online environment, an introduction 
to the task in which a general description of the task's content is described, its 
importance and its relation to the subject and the expected manner in which the 
learner is expected to act, clear work instructions that are understandable to the 
learner and a clear and detailed definition of the required product. 

General didactics (pedagogical knowledge) deals in: adaptation to the target audience, 
adaptation of sources to which the task refers, affording a solution for the 
heterogeneity of the students in the learning process, the readability and clarity of the 
language in the texts of the task or to which the task refers, inviting meaningful 
learning, encouraging creativity/originality of the learner, the interactivity of the task 
and the development of IT skills.Pedagogical content knowledge deals in congruity 
with the curriculum in the content field, and in ways for representing the information 
that are appropriate to the field of knowledge and the subject of the task. 

Technological knowledge and technological pedagogical content knowledge, i.e. 
implementation of teaching in an online environment deals in added value of the digital 
text and online environment, diverse means of illustration and expression, designing 
the task in a digital format, orientation and clarity of navigation, organization of the 
information in the digital format, and meeting the rules of ethics on the internet. 

The optional measures include originality and representations of the task, collaborative 
learning, high-order cognitive skills and affording a solution for the heterogeneity of 
the students and the use that can be made of the online task. The compulsory 
measures are the core of the task. In spite of the great importance which we attribute 
to the optional measures, we did not include these in the core at this stage, out of 
familiarity with the learning environment in the school and the teachers' mastery of the 
technologies. In the years 2001-2007, during which the tasks were constructed, the 
teachers did not use collaborative learning because web2 technology was not yet in 
use in the education system and platforms of collaborative learning had not yet been 
introduced in teachers' training curricula. 
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Criteria such as collaborative learning and development of high-order cognitive skills 
were therefore included in the optional measure. Most of the optional measures are 
another stratum, in addition to the core measures which are an essential element. For 
example, the pedagogical knowledge, which is compulsory, includes creation of new 
knowledge and creativity, which actually represent high-order cognitive skills and we 
regarded them as criteria that must appear in the compulsory part of the indicator. The 
criteria included in the optional measure of high-order cognitive skills offer other 
cognitive skills that can in our opinion appear in an online task but are not essential. 
The optional measures are: 

Originality and representations deal in the adaptation of the product to the 
heterogeneity of the students, the originality of the task and the absence of any 
stereotypes. 

Collaborative learning deals in the construction of shared knowledge, realization of 
shared instruments, shared learning product, open communication between learners, 
self-evaluation and collaborative of learning, and the active presence of the teacher 
during the course of collaborative learning. Learning at high-order cognitive skills and 
affording a solution for heterogeneity deals in high-order cognitive skills and open 
learning situations. 

Use of the online task deals in adaptation to learning in additional contexts, whether 
the resources needed for performing the task are standard, accessible and available, 
and whether accessibility exists for the disabled. 

Each criterion has three levels of performance standards: 1- low; 2- medium; 3- high. 

We also added a measure of an evaluator's evaluation. 

The research questions 

> Is the indicator that was developed for evaluating online learning tasks 
reliable? 

> Does the indicator that was developed evaluate the quality of the online 
task? 

METHODOLOGY 

The research instruments 

A digital evaluation indicator was developed and adapted for this research (see 
appendix 1), which was constructed based on the "megalithic model" (Rotem 2006). 
The megalithic model is intended to characterize and evaluate an online task according 
to foundation stones, covered by top stones. The foundation stones deal in the learning 
content, didactic realization, realization of online tools and digital text and the personal 
learning enabled by the task. The top stones deal in collaborative learning, high-order 
personal learning and the possibility of reconstructing and replicating the activity in 
other contexts. We performed several processes in order to adapt and validate the 
megalithic model: 

> Evaluating different indicators found in the literature. 

> Development and adaptation of the indicator based on the megalithic model 
and findings of the literature search. 

> Organization of the indicator according to compulsory and optional 
measures, and according to criteria and performance levels. The indicator 
was constructed such that is could be used for both research purposes and 
as an indicator for evaluating online tasks. 
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> Evaluating identical online tasks using the indicator (each task was 
evaluated by each of the researchers). 

> Improving and pinpointing the measures, criteria and performance levels 
that were found vague and were not understood identically by the 
investigators. 

> Additional evaluation of identical tasks by both researchers 

We examined content validity, i.e. whether the indicator meets criteria of common 
sense. Construct validity: a meticulous examination of a clear relation between the 
evaluation instrument and the theory on which the quality of a computerized task is 
based. The indicator that was developed contains 42 measures and criteria of four 
orders: 

> Thirty-one first order (criteria). 

> Seven second order (measures of the learning task, general pedagogical 
knowledge, pedagogical content knowledge, technological content 
knowledge (implementation of teaching in an online environment), 
originality and representations, collaborative learning, high-order cognitive 
skills and affording a solution for the heterogeneity of the students and use 
of the online task. 

> Three third orders (compulsory measures, optional measures and the 
evaluator's general evaluation). 

> A summarizing measure of the fourth order (summarizing score for the 
task). 

The two investigators evaluated online tasks using the digital indicator and tested the 
examined criteria according to customary characteristics of a quality online task in the 
literature. Each criterion in the indicator has three levels of performance: low, medium, 
and high. 

Testing the reliability of the indicator 

Testing the reliability of the digital indicator for characterizing and evaluating an online 
learning task was performed by comparing the evaluation scores of the investigators. 
We examined reliability using the method of reliability as stability - the extent to 
which repeated measurements, under identical conditions, led to similar results, inter¬ 
rater reliability, i.e. whether different evaluators obtained similar results. Each task 
that was evaluated within the framework of the study was evaluated by each of the 
investigators separately. Since this research is part of a more comprehensive research, 
the tasks were tested at two points of time in order to also test the improvement of the 
tasks over time. 

The research population 

Fifty-three online tasks were collected, which were developed by 14 high school 
teachers in different disciplines (sciences - biology and physics, social sciences, 
geography, civics and foreign language). The teachers all have a bachelor's degree in 
their major teaching subject, some have a master's degree and some also have a Ph.D. 

FINDINGS 

The indicator was found to be valid and reliable and can be used to evaluate 
improvement in the quality of the tasks over time. This was performed by analyses that 
examined the distribution of each dependent variable using measures of descriptive 
statistics - mean, median, standard deviation and range. The correlations between the 
second and third order evaluation measures were examined by calculating Pearson 
coefficients. 
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Focusing on these measures was performed because of the large amount of evaluation 
scores, and because these measures, by being aggregate scores of lower order 
measures, express these measures. 

Characterization of the Evaluation Tool - The Indicator 

Each evaluation measure was measured at two points of time (time I: 2001-2004; time 
II: 2005-2007) by both investigators. Two scores were calculated. Each score was 
calculated as the mean of the two ratings. A higher value in each score indicates better 
ability. Inter-rater reliability was calculated for each score at each point of time using a 
Pearson correlation between the two ratings and its correction using the Spearman- 
Brown equation. 

Table: 2 

Inter-rater reliability values for the evaluation measures 
(53 tasks) in an order and time section 


Measure evaluated 

Until 

2005 

After 

2005 

Compulsory statements 

0.98 

0.99 

Components of the learning task 

0.96 

0.99 

Framework of the task 

1 

0.99 

Prior knowledge 

0.96 

0.72 

Goals 

0.97 

0.95 

Description of the task 

0.96 

0.98 

Work instructions for the learner 



Definition of the learning products 


1 

General didactics (pedagogical knowledge) 

0.99 

0.99 

Adaptation to the target audience 



Adaptation of sources 

0.99 

0.90 

Affording a solution for the heterogeneity of the students 

0.92 

0.96 

Readability and clarity of language 

0.97 

0.65 

Inviting meaningful learning 

1 

0.93 

Encouraging creativity/originality 

1 

0.91 

Interactivity 

1 

1 

Development of IT skills 

0.90 

0.98 

Teaching method unique to the subject (pedagogical content 
knowledqe) 

0.99 

0.98 

Congruence with the curriculum 


0.91 

Representation of the information and learning methods 

1 

0.95 

Implementation of teaching in an online environment 

0.96 

0.98 

Added value to the digital text 

0.87 

0.93 

Diversity of means of illustration and expression 

0.96 

0.93 

Design of the task in a digital format 

0.84 

0.89 

Orientation and clarity of navigation 

0.89 

0.20 

Organization of the information in digital format 

0.99 

0.97 

Meeting the rules of ethics on the internet 

0.96 

0.96 

Optional statements 

1 

1 

Originality and representations 

1 

1 

Adaptation of the product to the heterogeneity of the students 

1 


Originality of the task 

1 

1 

Absence of stereotypes 



High-order cognitive skills and affording a solution for heterogeneity 

1 

0.99 

Cognitive skills 

1 

0.98 


Open learning situations 


1 
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Use of the online task 

1 

0.97 

Instructions for the teacher 



Adaptation to learning in additional contexts 

0.99 

0.96 

Resources for performing the task 

1 

1 

Access to individuals with disabilities 



General evaluation 

0.98 

0.94 

Final score for the task 

0.99 

0.99 


Note: Reliability was not calculated when no variance was found in one or both ratings. 


The reliability values for the two scores in each evaluation parameter are presented in 
table 2. It can be seen that for all scores, except orientation and clarity of navigation 
and level of readability and clarity of language at time II, the reliability level is 
standard (>0.70). A general score for each evaluation measure was not calculated as a 
mean of the two measurements beyond the measurement time, due to the low values 
of the reliability over time (stability). Reliability was calculated by the Pearson 
correlation between two administrations and the Spearman-Brown correction 
equation. However, it was found to be low and unacceptable for the scores of all 
measures (-0.95<r<0.66) 

The data in table 2 indicate high inter-rater reliability in the evaluation measures: in 
the compulsory statements (0.98, 0.99), the optional statements (1,1), the general 
evaluation (0.98, 0.94) and the final score for the task (0.99, 0.99). 

Evaluating the Quality of the Online Tasks 

Table: 3 presents' measures (mean and median) and the distribution measures 
(standard deviation and range) for the distribution of the dependent research variables 
- the evaluation measures. 


Table: 3 

Measure for the evaluation measures in an order and time section (53 tasks) 


Evaluation 

measure 

Mean 

Median 

Standard 
deviation SD 

Range (min-max) 


Time I 

Time II 

Time I 

Time II 

Time I 

Time II 

Time I 

Time II 

Compulsory 

statements: 

summarizing 

score 

71.43 

74.18 

71.37 

75.73 

7.14 

6.97 

(53-80.04) 

(57.55-84.50) 

Components of 
the learninq task 

2.20 

2.18 

2.17 

2.20 

0.18 

0.20 

(2-2.54) 

(1.94-2.59) 

Framework 
of the task 

2.93 

2.88 

3.00 

3.00 

0.27 

0.26 

(2-3) 

(2.17-3) 

Prior knowledqe 

1.11 

1.08 

1.00 

1.00 

0.24 

0.14 

(1-1.75) 

(1-1.42) 

Goals 

1.35 

1.22 

1.00 

1.00 

0.47 

0.43 

(1-2.25) 

(1-2.50) 

Description of 
the task 

1.83 

1.89 

2.00 

1.96 

0.62 

0.70 

(1-2.75) 

(1-3) 

Work 

instructions for 
the learner 

3.00 

3.00 

3.00 

3.00 

0 

0 

(3-3) 

(3-3) 

Definition of the 

learning 

products 

3.00 

2.99 

3.00 

3.00 

0 

0.04 

(3-3) 

(2.83-3) 

General didactics 

(pedagogical 

knowledge) 

2.04 

2.16 

2.08 

2.18 

0.21 

0.19 

(1.50- 

2.32) 

(1.69-2.47) 
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Adaptation to 
the target 
audience 

3.00 

2.99 

3.00 

3.00 

0 

0.03 

(3-3) 

(2.88-3) 

Adaptation of 
the sources 

2.70 

2.82 

3.00 

3.00 

0.61 

0.41 

(1-3) 

(1.50-3) 

Solutions for the 
heterogeneity of 
the students 

1.14 

1.42 

1.00 

1.33 

0.24 

0.40 

(1-1.67) 

(1-2) 

Readability and 
clarity of 
language 

2.68 

2.95 

3.00 

3.00 

0.46 

0.08 

(2-3) 

(2.75-3) 

Inviting 

meaningful 

learning 

2.48 

2.69 

2.50 

2.79 

0.42 

0.36 

(2-3) 

(2-3) 

Encouraging 

creativity/origin 

ality 

1.11 

1.07 

1.00 

1.00 

0.29 

0.16 

(1-2) 

(1-1.50) 

Interactivity 

1.20 

1.21 

1.00 

1.00 

0.37 

0.38 

(1-2) 

(1-2) 

Development of 
IT skills 

2.02 

2.07 

2.00 

2.00 

0.50 

0.53 

(1-2.75) 

(1-3) 

Teaching method 

unique to the 

subject 

(pedagogical 

content 

knowledge) 

2.19 

2.29 

2.20 

2.31 

0.20 

0.17 

(1.70- 

2.40) 

(1.85-2.58) 

Congruence with 
the curriculum 

3.00 

2.95 

3.00 

3.00 

0 

0.11 

(3-3) 

(2.67-3) 

Representation 
of the 

knowledge and 
methods of 
learning 

2.55 

2.68 

2.58 

2.75 

0.46 

0.32 

(2-3) 

(2-3) 

Implementation 
of teaching in an 
online 

environment 

2.18 

2.29 

2.17 

2.34 

0.38 

0.37 

(1.33- 

2.92) 

(1.50-2.81) 

Added value to 
the digital text 

2.49 

2.54 

2.58 

2.71 

0.63 

0.62 

(1-3) 

(1-3) 

Diversity of 
means of 
illustration and 
expression 

2.22 

2.41 

2.25 

2.50 

0.75 

0.53 

(1-3) 

(1-3) 

Designing the 
task 

in a digital 
format 

2.48 

2.53 

2.58 

2.79 

0.56 

0.55 

(1-3) 

(1.50-3) 

Orientation and 
clarity of 
navigation 

2.63 

2.78 

2.88 

2.83 

0.57 

0.27 

(1-3) 

(2.25-3) 

Organization of 
the information 
in 

digital format 

1.40 

1.80 

1.13 

1.75 

0.58 

0.83 

(1-3) 

(1-3) 

Meeting the 
rules of ethics 
on the internet 

1.83 

1.64 

1.83 

1.58 

0.76 

0.58 

(1-3) 

(1-2.67) 
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Optional 

statements: 

summarizing 

score 

71.93 

73.53 

74.63 

74.95 

6.08 

5.19 

(61.67- 

81.11) 

(62.82-83.33) 

Originality and 
representations 

2.27 

2.31 

2.33 

2.33 

0.14 

0.06 

(2-2.50) 

(2.11-2.33) 

Adaptation of 
the product to 
the 

heterogeneity of 
the students 

1.04 

1.00 

1.00 

1.00 

0.13 

0 

(1-1.50) 

(1-1) 

Originality of the 
task 

2.79 

2.95 

3.00 

3.00 

0.38 

0.18 

(2-3) 

(2.33-3) 

Absence of 
stereotypes 

3.00 

3.00 

3.00 

3.00 

0 

0 

(3-3) 

(3-3) 

High-order 
cognitive skills 
and affording a 
solution for 
heterogeneity 

1.89 

2.06 

2.00 

2.00 

0.53 

0.46 

(1-2.50) 

(1-3) 

Cognitive skills 

1.89 

2.07 

2.00 

2.00 

0.53 

0.47 

(1-2.50) 

(1-3) 

Open learning 
situations 

1.14 

1.02 

1.00 

1.00 

0.36 

0.05 

(1-2) 

(1-1.17) 

Use of the online 
task 

2.22 

2.20 

2.33 

2.23 

0.27 

0.19 

(1.33- 

2.33) 

(1.67-2.33) 

Instructions for 
the teacher 

1.00 

1.00 

1.00 

1.00 

0 

0 

(1-1) 

(1-1) 

Adaptation to 
learning in 
additional 
contexts 

2.74 

2.65 

3.00 

2.71 

0.58 

0.44 

(1-3) 

(1.67-3) 

Resources for 
performing the 
task 

2.93 

2.95 

3.00 

3.00 

0.27 

0.18 

(2-3) 

(2.33-3) 

General 

evaluation 

52.02 

52.06 

57.50 

55.56 

21.10 

19.99 

(1.67- 

71.67) 

(14.17-85) 

Final score of the 
task 

69.64 

71.83 

70.74 

73.80 

7.31 

6.89 

(50.47- 

78.97) 

(57.36-81.70) 


The data in Table: 3 indicate that the expected value in the distribution of many of the 
measures is around one from both ends of the evaluation scale (1 and 3). For example, 
the mean score for work instructions for the learner, defining the learning product, 
adaptation to the target audience, congruence with the curriculum, absence of 
stereotypes and originality of the task are located near or even on the upper limit of 
the measurement scale (3). 

In contradistinction, the mean score for prior knowledge, encouraging 
creativity/originality, adaptation of the product to the heterogeneity of the students, 
open learning situations, and instructions for the teachers is located near or even on 
the lower end of the measurement scale (1). 

Therefore, the standard deviation of these scores (the ones near 1) ranges from 0 to 
0.36 at most, the range is limited and ranges from 0 to 1 at most. A significantly higher 
distribution was found in scores located around the center of the measurement scale, 
such as meeting the rules of ethics on the internet (standard deviation 0.76 at time I 
and range 1-3), organization of the information in digital format (standard deviation 
0.83 and range 1-3) and general evaluation score (standard deviation 19.99 at time 
I and 21.10 at time II and range 1.67-71.67 and 14-17-85, respectively). 58 




Analysis of the Correlations Between 

The Second and Third Order Evaluation Measures 

The second order evaluation measures are components of the learning task, general 
didactics (pedagogical knowledge), teaching method unique to the subject 
(pedagogical content knowledge), implementation of teaching in an online 
environment, originality and representations, collaborative learning, high-order 
cognitive skills and affording a solution for heterogeneity and use of online tasks. 

The third order evaluation measures are compulsory measure, optional measures and 
the evaluator's general evaluation. 

The correlations between the second and third order evaluation measures were 
analyzed by calculating Pearson correlations. Table: 4 presents the correlations 
between the second order measures. 


Table: 4 

Correlations between second order evaluations 
measures at the two measurement times (53 tasks) 


Evaluation measure 

1 

2 

3 

4 

5 

6 

7 

1. Components of the 
learning task 


0.45 

0.55* 

0.36 

0.33 

0.46* 

0.56* 

2. General didactics 

(pedagogical 

knowledge) 

0.59* 


0.99*** 

0.77*** 

0.16 

0.10 

0.61** 

3. Teaching method 
unique for the subject 
(pedagogical content 
knowledge) 

0.58* 

0.98** 


0.76*** 

0.16 

0.11 

0.62** 

4. Implementation of 
teaching in an online 
environment 

0.01 

0.73** 

0.78*** 


0.20 

-0.13 

0.62** 

5. Originality and 
representations 

0.05 

0.17 

0.23 

0.16 


0.09 

0.80*** 

6. High-order 
cognitive skills and 
solution for 
heterogeneity 

0.55* 

0.25 

0.22 

-0.18 

.26 


0.09 

7. Use of the online 
task 

0.24 

0.75** 

0.76*** 

0.68** 

0.09 

-0.09 



* p<0.05, ** p<0.01, *** p<0.001 (one-way) 


Note: Correlations of time I are below the main diagonal and correlations of time II are 
above the diagonal. For example: The correlation between the components of the 
learning task and general didactics at time I is 0.59 and at time II it is 0.45. 

Table: 4 presents some very high correlations between the measures of general 
didactics (pedagogical knowledge), teaching method unique to the subject, 
(pedagogical content knowledge) and implementation of teaching in an online 
environment: general didactics (pedagogical knowledge) and teaching method unique 
to the subject (pedagogical content knowledge) (r=0.98, p<0.001 at time I and 
r=0.99, p<0.001 at time II), teaching method unique to the subject (pedagogical 
content knowledge) and implementation of teaching in an online environment (0.78 
and 0.76, respectively), general didactics (pedagogical knowledge) and 
implementation in an online environment (0.73 and 0.77, respectively). 59 
























This indicates a theoretical or operative overlapping between these measures. Very 
high correlations between variables can be due to two reasons: great overlapping in 
the theoretical definition of the two variables, or great overlapping in the measurement 
(operative definition) of the two variables. 

r 

High correlations were also found in the measure of use of the online task and in other 
measures, especially at time I, for example with unique teaching method for the 
subject (pedagogical content knowledge) (r=0.76). The measures high-order cognitive 
skills and affording a solution for heterogeneity as well as originality and 
representations were not found to have significant correlations with the other 
measures, except for the relation between high-order cognitive skills and affording a 
solution for heterogeneity and with components of the learning task (0.55 at time I 
and 0.46 at time II), and the relation between originality and representations and use 
of the online task at time II (0.80). 


Table: 5 

Correlations between the third order evaluations 
measures at the two measurement times (53 tasks) 


Evaluation measure 

1 

2 

3 

1. Compulsory 


0.42 

0.92*** 

2. Optional 

0.58* 


0.39 

3. General evaluation 

0.79*** 

0.50* 

— 


* p<0.05, ** p<0.01, *** p<0.001 (one-way) 

Note: Correlations of time I are below the main diagonal and correlations of time II are above 

the diagonal. 

It can be seen that the correlations between the third order evaluation measures 
(compulsory measures, optional measures and general evaluation) presented in table 5 
are mainly significant positive relations. The relation between the compulsory measure 
and the general evaluation score is highest (r=0.79, p<0.001 at time I and r=0.92, 
p<0.001 at time II). Here too this relation indicates very high overlapping between the 
two measures. 

DISCUSSION AND CONCLUSIONS 

Use of an online environment for teaching is a major part of the processes supposed to 
be taking place in the education system. This is the natural environment in which 
youths live and function, and the education system should therefore use this 
technology in a correct and wise manner. As mentioned by Salomon (2000), technology 
itself will not lead to a change, but will help in the realization of a new pedagogy. In 
light of the awareness of the educational potential that exists in the integration of 
technology in education, Salomon presents a vision in which technology will be at the 
disposal of pedagogy and will help in its realization: Technology will enable 
accessibility to information and will supply interactive and collaborative instruments 
and the teacher will create learning situations that exploit the educational potential of 
technology, situations that require coping with high-level tasks and teamwork. 

This article presented an indicator for evaluating online tasks that examines the 
pedagogical aspect which integrates technology and disciplinary contents (TPACK): 
technological pedagogical content knowledge (Mishra and Koehler 2006, 2009).The 
indicator was found to be valid and reliable and can be used to examine changes over 
time in TPACK among teachers. 
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The indicator for evaluating online tasks was adapted such that it would evaluate 
content knowledge, pedagogical knowledge, technological knowledge and 
technological pedagogical content knowledge -TPACK. These aspects were chosen after 
studying researches that demonstrate that online teaching will enable meaningful 
learning in which the learner is active and acquires high-order cognitive skills, such as 
making generalizations, asking questions, expressing a reasoned opinion, making 
comparisons or solving problems (Capper 2003; Herrington et al. 2005; Linn et al. 
2004). 

The combination between the pedagogical rationale and the technological tools 
enables exposure of the learners to rich and diverse information, enables dealing in 
complex contents that are relevant to the learners and enables experience in 
constructing rich and original products based on the newly acquired knowledge. This is 
learning that encourages cooperation while maintaining a meaningful educational 
dialogue with the teacher and with peers by means of the technological tools 
(Roschelle 2000; Lehtinen et al. 1998). 

Uniqueness of the Indicator for Evaluating Improvement In Tasks Over Time 

The tool was developed as an indicator with evaluation measures and performance 

criteria. The indicator contains three levels of evaluation measures. 

The first order evaluation measures are the criteria. The second order evaluation 
measures are the components of the learning task, general didactics (pedagogical 
knowledge), teaching method unique to the subject (pedagogical content knowledge), 
implementation of teaching in an online environment (technological knowledge and 
TPACK), originality and representations, collaborative learning, learning at high-order 
cognitive skills and affording a solution for heterogeneity and use of the online task. 
The third order evaluation measures are compulsory measures, optional measures and 
the general evaluation of the evaluator (see appendix 1). 

These three levels enable evaluating whether there is an effective combination of 
technology in pedagogy and contents in the online task and thus to evaluate the 
improvement in each of the following aspects and the integration between them: 
content knowledge, pedagogical knowledge, technological knowledge and 
technological pedagogical content knowledge. 

An indicator for evaluating an online task should also comprise an instrument in the 
hands of the teachers to improve and advance the next tasks which they construct (Or 
Meir 2005). 

When we examined other indicators that appear in the literature we saw that they do 
not afford sufficient resolution for our needs on knowledge in different fields 
(pedagogical knowledge), teaching method unique to the subject (pedagogical content 
knowledge) and implementation of teaching in an online environment (technological 
knowledge and technological, pedagogical content knowledge). The indicator which we 
used in the present research enables focusing on the various fields of knowledge and 
identification of the weak and strong points in these fields. 

This indicator is not only an indicator for evaluating an online task. It can also be used 
by the teachers as a tool which enables them to test their professional development in 
aspects of content knowledge, technological knowledge, pedagogical knowledge, and 
in combination turns them into experts in writing online tasks: in the field of 
technological pedagogical content knowledge. This is a digital indicator that enables 
obtaining a measure of the quality of the task immediately and immediately identifies 
the strengths and weaknesses of each task. A scale of three levels of performance 
enables differentiation of results at low sensitivity. 61 


We recommend examining expansion of the indicator to a scale of five levels of 
performance. For example, the existing three levels of performance in the indicator for 
the criterion "added value of the digital text and the online environment" are: 

> the digital text and the online environment have no added value for the task. 

> A constraint of the online/computerized tools for the task is apparent, or 
only some of the tools have an added value to learning. 

> The technological realization of digital tools and text has an added value to 
the learning process and to the advancement of learning. 

If we would convert the scale into a scale of five levels of performance, the choice 
between the criteria will be clearer and will afford a more accurate score. The following 
example illustrates the possibilities and sensitivities that a scale of five levels of 
performance affords to the criterion of "added value of the digital text and the online 
environment:" 


> the online task looks like a work sheet that can be printed. 

> Use of computerized tools is apparent, but there is no use of online tools at 
all - for example there is use of hypertext in the task pages but no use of 
the web at all. 

> A constraint of the online/computerized tools is apparent in the task - for 
example a forced use of the sources to which the task refers. 

> Only some of the tools have an added value to learning. 

> The technological realization of the digital tools and text has an added value 
in the learning process and in advancing learning. 

CONCLUSION 

The digital indicator which was developed enables immediate evaluation of online 
tasks. The indicator can show improvement in the tasks over time in different fields: 
content knowledge, pedagogical knowledge, pedagogical content knowledge and 
TPACK. The indicator also enables teachers to construct better online tasks by 
identifying foci of difficulty prominently and immediately. 

Thus, technology which is at the disposal of pedagogy helps in its realization since 
educated use of computerization in teaching may support meaningful teaching and 
learning and may comprise a lever for the teacher's coping with didactic, content and 
organizational issues (Linn et al. 2004). 
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Appendix No. 1: 

Indicator for evaluating the quality of online tasks 

Note: This is a digital indicator. This appendix presents only the text, without its digital 
advantages. 

Indicator for evaluating an online task 
Instructions for filling in the indicator: 

Type a number between 0-3 in column H. 1-3 according to the appropriate column and 
0 if the statement is not relevant. 

Type the personal evaluation of the task in cell E113. The evaluation will be a number 
between 0.00-3.00. 



Characteristics 
of the online 
task 

3 - high level 
of performance 

2 - medium 
level of 
performance 

1 - low level of 
performance 

Ranking 
1-3 or 0 
- not 
relevant 

Components of 
the learning 

task 

Framework of 
the task 

Are indicated: 

• Name of the 
school 

• Names of the 
writers of the 
task 

• Date of 
constructing 
the task 

• Subject of 
the task 

• Target 
audience 

• Schedule 

Are indicated: 

• Subject of 
the task 

• Schedule 

• Target 
audience 

There is no 
framework for 
the task, none 
of the required 
items appear 


Prior 

knowledge 

There is a 
detailing of the 
prior 

knowledge 
required for 
performing the 
task 

The required 
prior 

knowledge is 
mentioned, 
with no 
detailing 

Required prior 
knowledge is 
not mentioned 


Goals 

Learning goals 
in the field of 
content and 
skills of using 
an online 
environment 
are indicated 

Goals are 
indicated 
partially 

No goals are 
indicated 
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Description of 
the task 

There is an 
introduction in 
which: 

• There is a 
general 
description of 
the content 

• Its 

importance 
and relation 
to learning 
the subject is 
indicated 

• The way of 
work 

expected of 
the learner is 
indicated 

There is a 

partial 

introduction 

There is no 
introduction to 
the task 



Work 

instructions for 
the learner 

There are clear 
work 

instructions 
that are 
understood by 
the learner: 
general work 
instructions, 
and 

instructions for 
each activity 
that is found in 
the task at 
each of the 
stages of the 
task 

There are 
partial work 
instructions for 
the learner 
and/or they 
are not clear to 
the learner 

There are no 
work 

instructions for 
the learner 


Defining the 

learning 

product 

There is a 
definition of 
the learning 
product 

There are clear 
and detailed 
instructions for 
performing the 
product 

There is a 
partial 

description of 
the learning 
product and/or 
not sufficiently 
detailed 

There is no 
definition of the 
product 


General 

didactics 

(pedagogical 

knowledge) 

Adaptation to 
the target 
audience 

There is 
adaptation 
between the 
complexity of 
the task, the 
content, and 
the learning 
skills to the 
target 
audience 

There is partial 
adaptation to 
the target 
audience 

There is no 
adaptation to 
the target 
audience 


Adaptation of 
sources 

The sources to 
which the task 
refers are 
compatible 
with the target 
audience 

The sources to 
which the task 
refers are 
partially 
compatible 
with the target 
audience 

The sources to 
which the task 
refers are not 
compatible with 
the target 
audience or 
there is not 
referral to 
external 

sources 
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Affording a 
solution for the 
heterogeneity 
of the students 
in the learning 
process 

There is 
reference to 
the 

heterogeneity 
of the students 
in the 
teaching¬ 
learning 
methods and 
in the scope of 
learning 

There is partial 
reference to 
the 

heterogeneity 
of the students 

There is no 
reference to the 
heterogeneity 
of the students 


Level of 
readability and 
clarity of the 
language 

The texts in 
the task (or 
those to which 
the learner is 
referred) are 
readable and 
clear 

The texts in 
the task (or 
those to which 
the learner is 
referred) are 
reasonably but 
not sufficiently 
clear 

The texts in the 
task (or those 
to which the 
learner is 
referred) are 
difficult and 
cumbersome 


Inviting 

meaningful 

learning 

The activity 
invites 
creation of 

new 

knowledge or 
summarizes 
knowledge in a 
manner that is 
meaningful for 
the learner 

The activity 
partially 
invites 
creation of 
knowledge or 
summarizes 
for the learner 

The activity 
does not invite 
creation of new 
knowledge or 
meaningful 
summary 


Promoting 
creativity / 
originality of 
the learner 

The activity 
invites 

creativity and 
original ways 
of expression 

The activity 

partially 

invites 

creativity and 
originality 

The activity 
does not invite 
creativity and 
originality 


Interactivity 

There is 

referral to 

interactive 

activity of the 

learners 

among 

themselves 

(such as an 

online 

discussion) 

There are 
instructions for 
an action 
which the 
student is 
required to 
perform for 
maintaining an 
interaction 

There are 
partial 

instructions for 
an interactive 
activity 

There is no 
interactive 
activity 


Development 
of IT skills 

The task 

contains 

reference to 

technological 

know-how 

skills (locating 

information, 

reading 

information, 

writing, 

representing 

knowledge, 

presenting 

knowledge and 

merging texts) 

There is partial 
reference to 
technological 
know-how 
skills 

There is no 
reference to 
technological 
know-how skills 
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Teaching 
methods unique 
to the subject 
(pedagogical 
content 
knowledge) 

Congruence 
with the 
curriculum in 
the field of 
knowledge 

The contents 
of the task 
appear in the 
curriculum 

Only some of 
the contents of 
the task 
appear in the 
curriculum 

The contents of 
the task do not 
appear in the 
curriculum 


Representation 
of the 

information 
and ways of 
learning 

The ways of 
representing 
the 

information is 
suitable for the 
field of 

knowledge and 
the subject of 
the task 

Some of the 
ways of 
representing 
the knowledge 
are suitable for 
the field of 
knowledge and 
the subject of 
the task 

The ways of 
representing 
the knowledge 
are not suitable 
to the field of 
knowledge and 
the subject of 
the task 


Implementation 
of teaching in 
an online 
environment 

Added value of 
the digital text 
and the online 
environment 

The 

technological 
realization of 
the digital 
tools and text 
have an added 
value to the 
learning 
process and 
advancement 
of learning 

A constraint of 
the online / 
computerized 
tools for the 
task is 
apparent, or 
only some of 
the tools have 
an added value 
to learning 

There is no 
added value of 
the digital text 
and the online 
environment to 
the task 


Diverse means 
of illustration 
and expression 

The task 
contains 
diverse means 
of illustration 
such as: 

Pictures; 
organization / 
presentation of 
information in 
tables; 

illustrations by 
animations; 
simulations; 
movies / clips; 
audio 

information; 

maps 

The task 
contains few 
digital means 
of illustration 

The task makes 
no use of digital 
means of 
illustration 


Design of the 
task in digital 
format 

The design of 
the task is 
inviting, the 
pictures are 
relevant to the 
topic 

The design of 
the task is 
inviting but the 
pictures are 
not relevant to 
the topic of the 
task 

The design of 
the task is not 
inviting and the 
pictures are not 
relevant to the 
task 


Orientation 
and clarity of 
navigation 

It is easy to 
orient in the 
task (friendly 
navigation) 

The navigator 
in the task is 
not friendly 
enough 

There is no 
navigator in the 
task 


Organization 
of the 

information in 
digital format 

The 

organization of 
the 

information is 
compatible 
with the 
properties of 
the medium: 
links between 
parts of the 
task 

There are few 
links in the 
task between 
the different 
parts of the 
information 

There are no 
links between 
the different 
parts of the 
task 
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Meeting the 
rules of ethics 
on the internet 

The task meets 
the rules of 
ethics on the 
internet: 
preservation of 
copyrights and 
maintenance 
of privacy 

The task 
partially 
preserves 
copyrights and 
privacy 

The task does 
not meet the 
rules of proper 
ethics 


Originality and 
representations 

Adaptation of 
the product to 
the 

heterogeneity 
of the students 

There is 
reference to 
the 

heterogeneity 
of the students 
in the product 

There is partial 
reference to 
the 

heterogeneity 
of the students 
in the product 

There is no 
reference to the 
heterogeneity 
of the learners 


Originality of 
the task 

The task 
contains a 
significant part 
that was 
written by the 
composer of 
the task 

Most of the 
task is use of 
ready 

materials on 
the internet 

The task has 
only a referral 
to a ready task 
on the internet 


Absence of 
stereotypes 

There are no 
stereotypes of 
any kind: race, 
religion, 
gender, 
nationality and 
worldview 

The task 
contains hints 
at or it can be 
understood 
that there are 
stereotypes 

The task clearly 

contains 

stereotypes 


Collaborative 

learning 

Construction of 

common 

knowledge 

There exists a 
collaborative 
learning 
activity in the 
task 

There exists 
clear and 
explicit 

construction of 
the learners' 
role 

There is a 
collaborative 
learning 
activity in the 
task, but there 
is no clear and 
explicit 

construction of 
the learners' 
role in the 
collaborative 
activity 

There is no 
reference to 
collaborative 
learning activity 
in the task 


Realization of 

collaborative 

tools 

Collaboration 
is realized in 
the 

appropriate 
online tools 
such as WIKI, 
blogs, social 
network, etc. 

The 

collaborative 
activity is 
realized by 
partially online 
tools or is 
inappropriate 

The 

collaborative 
activity is not 
realized by 
targeted 
collaborative 
online tools 


Collaborative 

learning 

product 

A clear 
collaborative 
learning 
product is 
defined where 
every students 
has a 

possibility and 
obligation of 
expression 

A collaborative 
product is 
declared 
vaguely and 
not every 
student has 
the possibility 
and obligation 
of expression 

A collaborative 
product is not 
defined 
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Open 

communication 
between the 
learners 

The 

communication 
is open, 
transparent, 
and creates an 
atmosphere of 
openness and 
intimacy which 
encourages 
participation 

The activity is 
carried out 
with open 
communication 
between the 
learners, but 
there is no 
invitation to 
openness and 
intimacy 
without fear of 
judgment and 
criticism in the 
task 

There is no 
open 

communication 
between the 
learners, there 
is no 

encouragement, 
neither in the 
instruction nor 
in the 

illustration for 
open 

communication 


Self and 
collaborative 
evaluation of 
the learning - 
reflection 

The activity 
invites 
reflection by 
the learner 
about his work 
and the work 
of his peers in 
the group 

The activity 
invites 
personal 
reflection or 
reflection of 
peers 

The activity 
does not invite 
reflection by 
the learner 


Instructor / 
teacher is 
present and 
active during 
the 

collaborative 

learning 

An instructor / 
teacher is 
present during 
the 

collaborative 
online learning 
- responds 
during the 
learning 

There is partial 
/ little 

presence of an 
instructor / 
teacher during 
the 

collaborative 

learning 

There is no 
presence of an 
instructor / 
teacher during 
the activity 


High-order 
cognitive skills 
and solution for 
heterogeneity 

Cognitive skills 

The learning 
that is 
proposed in 
the task invites 
the 

development 
of Cognitive 
skills 

The learning in 
the task 
partially 
invites the 
development 
of Cognitive 
skills 

The learning in 
the task does 
not invite the 
development of 
learning skills 


Open learning 
situations 

The task 
invites open 
learning 
situations 

The task does 
not make wise 
/ proper use of 
open learning 
situations 

There are no 
open learning 
situations in the 
task 


Use of the 
online task 

Instruction for 
the teacher 

There exists a 
clear 

instructions 
sheet for the 
teacher 

There are 
partial and/or 
unclear 

instructions for 
the teacher 

There are no 
instructions for 
the teacher 



Adaptation to 
learning in 
additional 
contexts 

The task can 
be adapted to 
different 
target 
audiences 
and/or 
different 
learning topics 

The task can 
be partially 
adapted to 
different 
target 
audiences 
and/or 
different 
learning topics 

The task cannot 
be adapted to 
different target 
audiences or to 
different 
learning topics 
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Resources for 
performing the 
task 

The resources 
required for 
performing the 
task are 
standard, 
accessible and 
available 

Some of the 

resources 
required for 
performing the 
task are 
standard and 
available 

The resources 
required for 
performing the 
task are not 
standard and 
are not 
available 



Accessibility 
for people with 
disabilities 

The task meets 
the standard of 
accessibility 
for people with 
disabilities 

The task 
partially meets 
the standard of 
accessibility 
for people with 
disabilities 

There is no 
reference to the 
standard of 
accessibility for 
people with 
disabilities 



Table for concentrating the results 


Compulsory statements 

Components of the learning 
task 



General didactics 
(pedagogical knowledge 



Pedagogical content 
knowledge 



Implementation of teaching 
in an online environment 


Optional statements 

Originality and 
representations 



Collaborative learning 



High-order cognitive skills 
and solution for the 
heterogeneity of the students 



Use of the online task 



Net score (out of 100) 


Compulsory statements 

0.00 

0.00 

Optional statements 

0.00 

0.00 

General evaluation 



The score of the evaluation should be filled in on a scale ol 

F 0.00-3.00 
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